amavisd-new

amavisd-new is a high-performance interface between mailer (MTA) and content checkers: virus scanners, and/or SpamAssassin. It is written in Perl for maintainability, without paying a significant price for speed. It talks to MTA via (E)SMTP or LMTP, or by using helper programs. Best with Postfix, fine with dual-sendmail setup and Exim v4, works with sendmail/milter, or with any MTA as a SMTP relay. For Courier and qmail MTA integration there is a patch in the distributed package.

Introduction
News
Download
Documentation
Support
Contributed and related software
Features
Tips and FAQ
Security considerations

Introduction

amavisd-new is a high-performance and reliable interface between mailer (MTA) and one or more content checkers: virus scanners, and/or Mail::SpamAssassin Perl module. It is written in Perl, ensuring high reliability, portability and maintainability. It talks to MTA via (E)SMTP or LMTP protocols, or by using helper programs. No timing gaps exist in the design, which could cause a mail loss.

It is normally positioned at or near a central mailer, not necessarily where users' mailboxes and final delivery takes place. If looking for a per-user and low-message-rate solution to be placed at the final stage of mail delivery (e.g. called from procmail or in place of a local delivery agent), there may be other solutions more appropriate.

When calling of Mail::SpamAssassin (SA) is enabled, it calls SA only once per message regardless of the number of recipients, and tries very hard to correctly honour per-recipient preferences, such as pass/reject, check/nocheck, spam levels, and inserting spam-related mail header fields.

amavisd-new benefits from the use of Perl module Net::Server, which offers a fast pre-forked multichild process control. amavisd-new provides rfc2821-compliant SMTP server and client, a rfc2033-compliant LMTP server and client, and generates rfc3462/rfc3464-compliant (ex rfc1892/rfc1894) delivery (and non-delivery) status notifications.

This makes it suitable for mail anti-virus and/or anti-spam checking on a busy mail gateways that care for reliability and standards compliance.

amavisd-new grew out of amavisd(-snapshot) (which in turn is a daemonized version of amavis-perl), but through five years of development turned into a separate product, hardly resembling its origin. The code is several times the size of its predecessor, yet faster in throughput, richer in features, compliant to standards, includes optional support for spam detection, and makes virus scanning optional and easier to adjust/extend. Compatibility with helper programs from amavisd(-snapshot) is retained.

All modifications since the original amavisd done by Mark Martinec, with contribution of ideas, patches and reports from the amavis-users mailing list community and individuals.

News

2018-10-08: amavisd-new-2.11.1.tar.bz2 release

Old News

2016-04-26: amavisd-new-2.11.0.tar.xz release

2016-03-18: amavisd-new-2.11.0-rc1.tar.xz release candidate

2014-10-26: amavisd-new-2.10.1.tar.xz release

2014-10-22: amavisd-new-2.10.0.tar.xz release

2014-10-17: amavisd-new-2.10.0-rc2.tar.xz release candidate

2014-09-29: amavisd-new-2.10.0-rc1.tar.xz release candidate

2014-06-27: amavisd-new-2.9.1.tar.xz release

2014-05-09: amavisd-new-2.9.0.tar.xz release

2014-05-07: amavisd-new-2.9.0-rc2.tar.bz2 release candidate

2013-09-04: amavisd-new-2.8.2-rc1.tar.bz2 release candidate (never released)

2013-06-28: amavisd-new-2.8.1.tar.gz released

2013-04-27: amavisd-new-2.8.1-rc1.tar.gz release candidate

2012-06-30: amavisd-new-2.8.0.tar.gz has been released

2012-06-30: amavisd-new-2.7.2.tar.gz has been released, it is a bug-fix update over 2.7.1

2012-05-22: amavisd-new-2.8.0-pre7.tar.gz is a preview of the next release

2012-04-29: amavisd-new-2.7.1.tar.gz has been released, it is a bug-fix update over 2.7.0

2012-04-29: amavisd-new-2.8.0-pre6.tar.gz is a preview of the next release

2012-04-10: amavisd-new-2.7.1-rc1.tar.gz is a release candidate for a bug-fix update on 2.7

2012-03-09: amavisd-new-2.8.0-pre4.tar.gz is a preview of the next release

2011-07-01: amavisd-new-2.7.0.tar.gz is a long-awaited features release

2011-05-19: amavisd-new-2.6.6.tar.gz is a maintenance update of a 2.6 branch, backporting all bug fixes from the 2.7.0-pre* development cycle

2011-05-18: amavisd-new-2.7.0-rc1.tar.gz pre-release

2011-03-07: Mailing list has been moved from SourceForge to amavis.org. The new posting address is amavis-users@amavis.org . Only posts from subscribed members are accepted, as before. The announcement is archived at https://lists.amavis.org/pipermail/amavis-users/2011-March/000005.html . The MARC archive of the previous mailing list is continuing to follow the new list, other third-party archives are no longer being updated.

2011-05-13: amavisd-new-2.6.6-rc1.tar.gz is a maintenance update of a 2.6 branch, backporting all bug fixes from the 2.7.0-pre* development cycle

2011-04-12: amavisd-new-2.7.0-pre15.tar.gz pre-release

2011-04-07: amavisd-new-2.6.5.tar.gz is a maintenance update of a 2.6 branch, backporting all bug fixes from the 2.7.0-pre* development cycle

2011-04-02: amavisd-new-2.6.5-rc2.tar.gz release candidate

2011-03-31: amavisd-new-2.6.5-rc1.tar.gz release candidate

2011-02-03: amavisd-new-2.7.0-pre14.tar.gz pre-release

2011-01-25: amavisd-new-2.7.0-pre13.tar.gz pre-release

2010-12-24: amavisd-new-2.7.0-pre12.tar.gz pre-release

2010-12-18: amavisd-new-2.7.0-pre11.tar.bz2 pre-release

2010-11-15: amavisd-new-2.7.0-pre9.tar.gz pre-release

2010-10-28: amavisd-new-2.7.0-pre8.tar.gz pre-release

2010-08-31: amavisd-new-2.7.0-pre7.tar.gz pre-release

2010-06-03: amavisd-new-2.7.0-pre5.tar.gz pre-release

2010-04-25: amavisd-new-2.7.0-pre4.tar.gz pre-release

2009-06-25: amavisd-new-2.6.4.tar.gz released

2009-06-19: amavisd-new-2.6.4-rc2.tar.gz release candidate

2009-06-12: amavisd-new-2.6.4-rc1.tar.gz release candidate

2009-04-22: amavisd-new-2.6.3.tar.gz released

2009-04-18: amavisd-new-2.6.3-rc2.tar.gz release candidate

2009-04-15: amavisd-new-2.6.3-rc1.tar.gz release candidate

2008-12-15: amavisd-new-2.6.2.tar.gz released

2008-12-06: amavisd-new-2.6.2-rc2.tar.gz release candidate

2008-11-20: amavisd-new-2.6.2-rc1.tar.gz release candidate

2008-11-14: amavisd-new-2.6.2-pre1.tar.gz pre-release

2008-06-29: amavisd-new-2.6.1.tar.gz released

2008-06-24: amavisd-new-2.6.1-rc1.tar.gz release candidate

2008-04-23: amavisd-new-2.6.0.tar.gz has been released

2008-04-19: amavisd-new-2.6.0-rc2.tar.gz release candidate

2008-03-19: amavisd-new-2.6.0-rc1.tar.gz release candidate

2008-03-12: amavisd-new-2.5.4.tar.gz maintenance version

2007-12-30: amavisd-new-2.6.0-pre3.tar.gz pre-release

2007-12-12: amavisd-new-2.5.3.tar.gz maintenance version

2007-06-27: amavisd-new-2.5.2.tar.gz

2007-05-31: amavisd-new-2.5.1.tar.gz

2007-05-23: A security vulnerability in a file(1) utility version 4.20 has been found. Note that this is not the same issue as CVE-2007-1536. The problem is fixed in version 4.21. This program is being used by amavisd-new even when virus scanning is disabled, so it is heartly recommended to use the most recent version (currently at 4.21), available from ftp://ftp.astron.com/pub/file.

See the FreeBSD-SA-07:04 security advisory which applies to a file(1) utility distributed with the operating system.

2007-04-23: amavisd-new-2.5.0.tar.gz

2007-03-22: An exploitable security problem in file(1) utility version 4.19 and older has been found by Jean-Sébastien Guay-Leroux. The problem is fixed in version 4.21 (partly fixed in 4.20). This program is being used by amavisd-new even when virus scanning is disabled, so it is heartly recommended to use the most recent version (currently at 4.21), available from ftp://ftp.astron.com/pub/file.

2005-06-24: MailZu is a quarantine management interface for amavisd-new, created by Samuel Tran and Brian Wong (beware, the domain MailZu.net no longer belongs to the project!)

Security warning

The amavisd-new uses several external programs and Perl modules for its operation. If there are security vulnerabilities in them, the whole setup might be affected. The possible damage is limited to what a non-privileged UID can accomplish in normal setups, and can further be limited using a chroot setup. Please see the Security considerations section below for additional information.

It is always a good idea to use fairly recent versions of external programs and external Perl modules. Some of the noteworthy known security problems are:

utility program file(1):: An explitable security problem in version 4.19 and older has been found by Jean-Sébastien Guay-Leroux, and a similar security problem in 4.20 found by Colin Percival. The problem is fixed in file(1) version 4.21. This program is being used by amavisd-new even when virus scanning is disabled, so it is heartly recommended to use the most recent version (currently at 4.21), available from ftp://ftp.astron.com/pub/file .
uulib:: An exploitable integer overflow leading to a buffer overflow was discovered in versions of uulib as distributed with Perl module Convert::UUlib older than version 1.05. Please use the most recent version of this module, which at the time of writing is 1.06. CVE-2005-1349
LHa dearchiver:: A LHa buffer overflows and directory traversal problem was described on the [Full-Disclosure] mailing list. Please use the patched version or use the latest version if available for your OS.
zoo: Stack-based exploitable buffer overflow in the fullpath function in misc.c for Zoo 2.10 and earlier: CVE-2006-0855
zoo: Zoo file decompression infinite loop DoS (zoo 2.10, unzoo.c): CVE-2007-1669, CVE-2007-1673, zoo advisory 2007-05-04

The use of each external decoding program can be disabled in file amavisd.conf by removing entries from the list @decoders, or in older versions or amavisd-new by removing assignment to corresponding variables (e.g. $lha, $unarj, $unrar, $zoo, ...) or setting them to undef, or just not having the named external program present on the $path.

Download

Home web site in Slovenia (at J. Stefan Institute):: => https://www.ijs.si/software/amavisd/
Mirror web site in Denmark (courtesy of catpipe Systems ApS):: => http://mirrors.catpipe.net/amavisd-new/
Mirror web site in Sweden (courtesy of Mainloop, Stockholm):: => http://mirror.mainloop.se/amavisd/
Mirror web site in France/Paris (courtesy of Cádrat Net):: => http://mirror.cedratnet.com/amavisd-new/
Mirror web site in the Netherlands/Hilversum (courtesy of Publieke Omroep Internet Services):: => http://mirror.omroep.nl/amavisd-new/
Mirror web site in Germany (courtesy of German Postfix Community):: => http://amavisd.de.postfix.org/

Most recent versions

amavisd-new-2.11.1.tar.bz2 (md5 of tar.bz2): Most recent minor patch release, please see RELEASE NOTES
amavisd-new-2.11.0.tar.xz (md5 of tar.xz); or bz2: Previous release
amavisd-new-2.10.1.tar.xz (md5 of tar.xz); or bz2: Previous release
amavisd-new-2.10.0.tar.xz (md5 of tar.xz); or bz2: Old release
amavisd-new-2.9.1.tar.xz (md5 of tar.xz); or bz2: Old release
amavisd-new-2.8.1.tar.gz (gz md5 sum); or xz, bz2: Old release

The most recent stable version is also accessible through a soft link at: https://www.ijs.si/software/amavisd/amavisd-new.tar.gz

Announcements about new releases are posted on the amavis-users mailing list, and on the Freshmeat project page, which also accepts subscriptions to announcements.

All versions

For a web-browsable Mercurial repository of all the past versions of amavisd, reaching all the way back to the origins of the project (AMaViS, amavis-perl, amavisd-snapshot, amavisd-new) please see the http://mirrors.catpipe.net/amavisd-new/hgweb/, maintained by Phil Regnauld (of catpipe.net).

amavisd-new-2.11.0.tar.xz (also known as amavisd-new-20160426) (xz md5 sum).
amavisd-new-2.10.1.tar.xz (also known as amavisd-new-20141025) (xz md5 sum).
amavisd-new-2.10.0.tar.xz (also known as amavisd-new-20141022) (xz md5 sum).
amavisd-new-2.9.1.tar.gz or amavisd-new-2.9.1.tar.xz (also known as amavisd-new-20140627) (gz md5 sum).
amavisd-new-2.9.0.tar.gz or amavisd-new-2.9.0.tar.xz (also known as amavisd-new-20140509) (gz md5 sum).
amavisd-new-2.8.1.tar.gz or amavisd-new-2.8.1.tar.xz (also known as amavisd-new-20130628) (gz md5 sum).
amavisd-new-2.8.0.tar.gz or amavisd-new-2.8.0.tar.xz (also known as amavisd-new-20120630) (gz md5 sum).
amavisd-new-2.7.2.tar.gz or amavisd-new-2.7.2.tar.xz (also known as amavisd-new-20120629) (gz md5 sum).
amavisd-new-2.7.1.tar.gz or amavisd-new-2.7.1.tar.xz (also known as amavisd-new-20120429) (gz md5 sum).
amavisd-new-2.7.0.tar.gz or amavisd-new-2.7.0.tar.xz (also known as amavisd-new-20110701) (gz md5 sum)
amavisd-new-2.6.6.tar.gz (also known as amavisd-new-20110518) (md5 sum)
amavisd-new-2.6.5.tar.gz (also known as amavisd-new-20110407) (md5 sum)
amavisd-new-2.6.4.tar.gz (also known as amavisd-new-20090625) (md5 sum)
amavisd-new-2.6.3.tar.gz (also known as amavisd-new-20090422) (md5 sum)
amavisd-new-2.6.2.tar.gz (also known as amavisd-new-20081215) (md5 sum)
amavisd-new-2.6.1.tar.gz (also known as amavisd-new-20080629) (md5 sum)
amavisd-new-2.6.0.tar.gz (also known as amavisd-new-20080423) (md5 sum)
amavisd-new-2.5.4.tar.gz (also known as amavisd-new-20080312) (md5 sum)
amavisd-new-2.5.3.tar.gz (also known as amavisd-new-20071212) (md5 sum)
amavisd-new-2.5.2.tar.gz (also known as amavisd-new-20070627) (md5 sum)
amavisd-new-2.5.1.tar.gz (also known as amavisd-new-20070531) (md5 sum)
amavisd-new-2.5.0.tar.gz (also known as amavisd-new-20070423) (md5 sum)
amavisd-new-2.4.5.tar.gz (also known as amavisd-new-20070130) (md5 sum)
amavisd-new-2.4.4.tar.gz (also known as amavisd-new-20061120) (md5 sum)
amavisd-new-2.4.3.tar.gz (also known as amavisd-new-20060930) (md5 sum)
amavisd-new-2.4.2.tar.gz (also known as amavisd-new-20060627) (md5 sum)
amavisd-new-2.4.1.tar.gz (also known as amavisd-new-20060508) (md5 sum)
amavisd-new-2.4.0.tar.gz (also known as amavisd-new-20060402) (md5 sum)
amavisd-new-2.3.3.tar.gz (also known as amavisd-new-20050822) (md5 sum)
amavisd-new-2.3.2.tar.gz (also known as amavisd-new-20050629) (md5 sum)
amavisd-new-2.3.1.tar.gz (also known as amavisd-new-20050509) (md5 sum)
amavisd-new-2.3.0.tar.gz (also known as amavisd-new-20050424) (md5 sum)
amavisd-new-2.2.1.tar.gz (also known as amavisd-new-20041222) (md5 sum)
amavisd-new-2.2.0.tar.gz (also known as amavisd-new-20041102) (md5 sum)
amavisd-new-2.1.2.tar.gz (also known as amavisd-new-20040906) (md5 sum)
amavisd-new-2.1.1.tar.gz (also known as amavisd-new-20040824) (md5 sum)
amavisd-new-2.1.0.tar.gz (also known as amavisd-new-20040815) (md5 sum)
amavisd-new-20040701.tar.gz (also known as amavisd-new-2.0) (md5 sum)
amavisd-new-20030616-p10.tar.gz (md5 sum)
amavisd-new-20030314-p2.tar.gz (md5 sum)
amavisd-new-20021227-p2.tar.gz (md5 sum)
amavisd-new-20021116.tar.gz (md5 sum): (a new features release, its mandatory patches: p1, p2, p3, p4, p5)
amavisd-new-20020630.tar.gz (md5 sum): (mostly a development and new features release)
amavisd-new-20020517.tar.gz (md5 sum): (first version with SpamAssassin (optional) support)

Ports and Packages

There are some packaged versions available, provided and supported only by their respective authors/maintainers. Some are recent and updated frequently, others are pretty much out of date.

FreeBSD port: in the System security software ports section (/usr/ports/security/amavisd-new), maintained by Michael Scheidell and Gábor Kövesdán. The latest port is based on amavisd-new-2.7.0;
NetBSD port: wip/amavisd-new has been moved to pkgsrc as security/amavisd-new, maintained by Julian C. Dunn, based on amavisd-new-2.7.0;
OpenBSD port: in the mail software ports section (/usr/ports/mail/amavisd-new), maintained by Giovanni Bechis. The latest port is based on amavisd-new-2.7.0;
OpenPKG RPM: cross-platform RPM-based Unix software packaging; current: ftp://ftp.openpkg.org/current/SRC/ is based on amavisd-new-2.6.4, CVS directory: http://cvs.openpkg.org/dir?d=openpkg-src/amavisd
Solaris CSW package: by Ihsan Dogan, available at http://www.opencsw.org/packages/amavisd_new is based on 2.6.4
Linux: Red Hat/Fedora Apt RPM packages amavisd-new: by Dag Wieërs, available at http://dag.wieers.com/packages/amavisd-new/; based on 2.6.6
Linux: Fedora RPMS/SRPMS amavisd-new: by Lukasz Trabinski, available at ftp://ftp.wsisiz.edu.pl/pub/Linux/rpms/Fedora-9/amavisd-new/; based on 2.6.1
Linux: SuSE RPM: available at ftp://ftp.norrbring.com/pub/linux/inst-source/, provided by Anders Norrbring, Norrbring Consulting; based on version 2.5.2
Linux: Gentoo ebuild: in the net-mail category (/usr/portage/net-mail/amavisd-new). The latest 'unstable' ebuild is maintained by Christian Zoffoli, Sune Kloppenborg Jeppesen and other contributors;
Linux: Debian package amavisd-new: maintained by Henrique de Moraes Holschuh and Brian May, available at: http://packages.debian.org/sid/amavisd-new; based on 2.6.4;
Linux: Mandriva Linux: by Giuseppe Ghibò, available at Mandriva Linux SRPMS as amavisd-new-*mdk.src.rpm; see also http://www.joeghi.com/amavisd-new/; based on 2.6.3

There may be other packaged version around, please let me know.

Note that packaged versions may not be based on the most recent version of amavisd-new.

Documentation

Besides this web page at https://www.ijs.si/software/amavisd/ (or mirrors), and the assorted bits and pieces of new documentation, the following files comprise the amavisd-new documentation. The web page documents may be more recent than the documentation distributed with the program.

amavisd-new-docs.html
RELEASE NOTES
AAAREADME.first
LICENSE (GNU General Public License)
INSTALL
AMAVIS-MIB
LDAP.schema
README.postfix, by Patrick Ben Koetter
README.postfix.old
README.exim_v4
README.exim_v4_app
README.exim_v4_app2
README.exim_v3
README.exim_v3_app
README.sendmail-dual, by Mark Martinec
README.milter (sendmail with milter), by Rob MacGregor
README.sendmail (non-milter, old), by Rainer Link (this setup is no longer recommended)
README.courier
helper-progs/README (only of interest with sendmail-milter and Exim v3)
README.chroot
README.customize
README.lookups
README.sql
README.sql-mysql
README.sql-pg
README.performance
README.policy-on-notifications
README.protocol
TODO
Macintosh.tar.gz, Mac OS X docs, by D. Walsh

How-to

Debian anti-spam anti-virus gateway email server using Postfix, amavisd-new, SpamAssassin, Razor, DCC, Pyzor and ClamAV HOWTO, by Gary V.; inspired by a document originally created by Scott L. Henderson, but rewritten to reflect a Debian installation and contains a considerable amount of additional information
Configuring clamav with amavisd-new, by Gary V.
Fairly-secure anti-spam gateway using OpenBSD, Postfix, amavisd-new, SpamAssassin, Razor and DCC, by Scott Vintinner; describes chrooted environment for amavisd-new and other components
Implementing a SPAM and virus scanning mail server using Red Hat Linux 8.0 and amavisd-new (using Postfix) (PDF), by Ralf Spenneberg
Creating a spamfilter relay server, by Scott L. Henderson
How to install Postfix, Amavis, ClamAV, and SpamAssassin on Debian Linux, by Tobias Rice
Gentoo mailfiltering gateway guide, by Sune Kloppenborg Jeppesen
Postfix and SpamAssassin, by Grzegorz Czapliñski, an article in Dæmon News, 2003-09, describing a setup with Sophos (or other) anti-virus, Postfix and Amavisd-new with SpamAssassin on a *BSD system
Configuring an Open Source Mail Gateway, by David Handermann, an article in freshmeat.net tutorials, 2003-03
Adding ClamAV Anti-Virus to an Anti-SPAM Gateway, by Kris Nosack
RadicalSpam -- Plateforme Anti-Spam et Anti-Virus Open Source sur Linux (Postfix, SpamAssassin, Razor, DCC, chroot-ed Amavisd-new, ClamAV, Bind, Postgrey, Rbldnsd, Nagios), by Stéphane Rault
Sécurité du service de courrier électronique : amavisd-new, by Philippe Latu (Linux France)
Mail gateway med Sendmail, Amavisd-new, ClamAV och SpamAssassin (in Swedish), by Johannes Petersson
Postfix-Cyrus-Web-cyradm-HOWTO, by Luc de Louw
Exim, Amavis, Qpopper with TLS+MySQL Auth Mini How-To, by Jani Reinikainen
amavisd-new + qmail, by Martin Solciansky
amavisd-new + qmail dual MTA setup, by Martin Solciansky
qmail-amavisd integration -- a "one-and-half"-MTA setup, by Alex Povolotsky
ISP Mailserver Solution Howto, by Martin List-Petersen
Kolab + amavisd-new (with Postfix) + SpamAssassin + ..., by Stephan Buys
Setting Up An Anti-SPAM Gateway, by Sam Hart
PerlStalker's SysAdmin Notes and Tools: Courier+amavisd-new, by Randall B. Smith

Support

Check the amavis-users mailing list. Its archives are at https://lists.amavis.org/pipermail/amavis-users, and at the MARC archive: http://marc.theaimsgroup.com/?l=amavis-user . Other third-party archives are no longer updated: http://groups.google.com/group/mailing.unix.amavis-user/topics?lnk=srg

For questions about packaged versions please contact their maintainers and/or their bug-tracking mechanisms.

Contributed and related software

amavisd-milter (by Petr Rehor): is a sendmail milter for amavisd-new version 2.2.0 and above which use the new AM.PDP protocol;
policyd v2 (by Nigel Kukard): version 2 of policyd allows integration with amavisd-new by overriding policy banks just before processing and allows finger grained control of the policy banks;
MailZu (by Samuel Tran, Brian Wong, and others): is a simple and intuitive quarantine management interface for amavisd-new; (beware, the domain MailZu.net no longer belongs to the project!) at SourceForge;
Artica for Postfix (by David Touzeau): a full Postfix Management console;
Modoboa (formerly named MailNG), by Antoine Nguyen: web-based interface and management system for virtual domains hosting, with a module to handle amavisd-new SQL quarantine;
Zentyal (formerly eBox Platform): a Linux Small Business Server that integrates Amavisd-new in the mailfilter module;
wblist (by James Bourne): is a web based interface to the amavisd-new SQL-based policy database, allowing users to edit their white-/black- list;
myAmavis (by Stefan Palme): is a web frontend for the SQL database that can be used by amavisd-new for policy lookup and storage;
Maia Mailguard (by Robert LeBlanc): is a web-based interface and management system for amavisd-new. Written in Perl and PHP, Maia Mailguard gives end-users control over how their mail is processed by virus scanners and spam filters, while giving mail administrators the power to configure site-wide defaults and limits;
Horde SAM (by Max Kalika): is a per-user SpamAssassin, whitelist and blacklist manager which now has native support for amavisd-new policies and attributes stored in an SQL server;
AmavisNewSQL is a SquirrelMail Plugin project (by Jared Watkins): It lets users change a pre-defined set of SpamAssassin settings when those settings are stored in a SQL database rather than a config file. It also allows you to use a quarantine database for questionable email messages. It was designed with enterprise use in mind, and differs from already existing plugins in that it works with amavisd-new rather than SpamAssassin directly.
WebAvis (by Jérôme Schell): is a Web frontend to amavisd-new written in PHP. It allows owners of a mail account to manage their amavisd-new parameters, like spam scores, white/black lists and filter behavior;
PostVis Admin (by Roger Smith): is an easy to use administration tool written in PHP for amavisd-new and Postfix. The main backend is MySQL for quarantine and user management;
OpenVISP Admin (by Xavier Beaudouin): is a fork of Postfix Admin (PHP-based) that handles some amavisd-new options and greylisting through SQL database;
ClamAV webmin module: includes an interface to the amavis quarantine to search/view/delete/re-inject virus infected and spam-tagged mail;
quarReminder (by BJ Dierkes): is a small PHP program that queries an amavisd-new SQL database and sends a list of messages in quarantine to email users;
process_bsmtp.pl (by Peter Collinson): feeds quarantined messages (in BSMTP format) to SQL database;
logwatch modules (by Mike Cappella): parses amavisd-new and Postfix logs, producing reports;
amavis-stats (originally written by Mark Lawrence, since 0.1.14 c/o Dale Walsh): is a simple statistics generator based on rrdtool. It produces graphs of infections from amavisd-new log entries broken down by virus. The RRD files are created and updated by a perl script, graphs are generated by a php script.
amavis-blocked (by Uwe S. Fuerst): a log file parser for amavisd-new 2.x, written in Perl
mailgraph (David Schweikert and others): collects data (into RRD) and plots virus and spam blocked by amavisd-new;
OpenVISPStats (a.k.a. OVS) (by Xavier Beaudouin): is a fork of MailGraph and Couriergraph; collects data (into RRD) and plots charts;
amavistat-new (by Marcus Schopen): is a modification of AmaviStats (by Heath Robinson) to work with amavisd-new.
amavislogsumm (by Stefan Jakobs, updated version of amavislogsumm based on earlier work by Matt Egan, originally by Sascha Hüdepohl): analyse amavisd-new logfiles
PheTail (apparently no longer available on-line, here is the phetail-0.1.tar.gz), (by Jesper Nøhr): continuously parses amavisd-new log file (tail) and feeds information about encountered viruses and spam into SQL database
ClamAV: Clam AntiVirus - an open source AV scanner
Sophie (by Vanja Hrustic, maintained by Richard Baldry): daemonised Sophos virus engine

Features

-- general

written in Perl for better maintainability, portability, no risk of buffer overruns or invalid pointers; critical code paths are well optimized;
a daemonized pre-forking server, saving on startup time; designed with high throughput in mind;
security conscious, runs under a dedicated user id (not root and not mail), can run in a chroot jail (including SpamAssassin and external programs);
thorough error checking, informative error reporting, fail-safe failure modes;
does not lose or corrupt mail when unpredictable things happen, including I/O errors, disk full conditions (to the extent of underlying Perl/file-system/OS ability to detect errors), or a machine crash; the responsibility for mail always stays with MTA;
does not let mail pass unchecked when unpredictable things happen or when mail is too big (with the exception that a mail size limit for spam checking can be specified); such conditions cause temporary failure to be reported to MTA, mail stays in MTA queue;
does not load entire mail into memory, so there are no arbitrary size limitations and out-of-memory conditions while transfering, decomposing, virus scanning, quarantining (including SQL quarantining of arbitrary size mail); an exception is mail passed to SpamAssassin, which, due to the way SA works, needs to be in memory, but a size limit can be specified above which SA is not called, or (starting with 2.6.3) SpamAssassin can still be called but be given a truncated message;
supports optional verification of DKIM and DomainKeys signatures regardless of mail size (even for mail not passed to SpamAssassin); valid signature can load a policy bank (e.g. customized per-sender configuration, based on a proven signer's signature), or can provide reputation data;
supports optional DKIM signing, with plenty of flexibility in choosing a suitable signature for multi-domain sites (author or third-party signatures); if desired, all other checks can be disabled and amavisd used as a DKIM signing service only;
DKIM and DomainKeys friendly - does not break signed mail, including keeping its integrity in a copy passed to SpamAssassin, so that SA plugins Mail::SpamAssassin::Plugin::DKIM can be reliably used without causing false positives;
optionally calls one or more anti-virus scanners - the current list includes entries for more than 40 AV scanners and is easily extended, see amavisd.conf file for the list;
optionally checks MIME types, file names and content types of decoded mail parts (content classifications are provided by a file(1) utility) against a list of banned names and content types;
optionally checks mail header for invalid characters and some other common violations of rfc2822, and produce informative bounces or notifications;
optionally calls Perl module Mail::SpamAssassin to check for spam, with optional hard or soft blacklisting/whitelist (regarding spam) of envelope sender address;
pen pals soft-whitelisting feature (since 2.4.2) reduces spam score on replies to previous correspondence sent from a local user (requires logging to SQL to be enabled);
bounce killer feature (requires pen pals SQL logging) checks a header section attached to received non-delivery status notifications, and discards bounces to fake mail which do not refer to our genuine outgoing mail;
may modify mail headers, but it doesn't modify mail body, even if configured to call SpamAssassin (with an exception that the 2.0 defanging feature can wrap original mail into a MIME wrapper). Starting with version of 2.5.0 there is an interface to external utilities altermime or Anomy Sanitizer, which allows for per-recipient sanitation of mail body or adding disclaimers.
versatile lookup mechanisms (ACL, hash, regexp lists, SQL, LDAP);
true per-recipient handling of most settings, even for multi-recipient messages;
modular (package namespaces), yet contained in a single file; only the required modules are compiled;
natively speaks SMTP or LMTP on the client and/or server side (with TLS if required), no need for potentially unreliable or slow interfacing with forks/pipes from scripts;
works best with MTAs which interfaces a content filter via SMTP (or LMTP), such as Postfix, Exim v4, or dual (any) MTA setup (all features available, fast), but can also interface with sendmail/milter, Courier, qmail, or other MTAs using helper programs;
supports client-side LMTP since version 2.5.0, which allows setting up amavisd as a LMTP-to-LMTP filter, feeding a local delivery agent such as Cyrus; note that such a setup does not check outgoing mail and does not allow Pen pals feature to be used, and is only useful for sites with specific needs;
standards compliant: adheres tightly to a bunch of RFC specifications - see release notes for details;
supports Delivery Status Notifications (DSN) extension to the SMTP protocol (RFC 3461) and notification messages as required by RFC 3462 and RFC 3464 (since 2.4.0);
quarantining to SQL, to files, or to mailboxes;
logging to syslog (or a file), and optionally to SQL (where the database is up-to-date and consistent at any time);
does not duplicate features that a decent MTA or SpamAssassin already provides;
does not rely on private data structures or private interfaces of an MTA.

-- performance

pre-forked reusable children managed by Net::Server Perl module, saving on process creations and startup latency;
significant speedups (25% with fast AV scanner compared to amavisd-snapshot-20020300 with the same AV scanner) in SMTP relay configuration; directory and temporary file reuse;
persistent connections to certain AV scanners, e.g. Sophie and Trophie; or directly callable AV scanner via Perl module to access Sophos engine: SAVI-Perl, or via Mail::ClamAV for ClamAV access;
cache of the recent body message digests (MD5) -- significant speedup for mailing list bursts, or equal-contents bursts of spam or viruses;
calls spam scanner (and virus scanners) once per message regardless of the number of recipients, gaining 50% speedup on SA calls on the average for free (depending on the average number of recipients per message), while still obeying most per-recipient settings, splitting mail if necessary;
when configured to call Mail::SpamAssassin (this is optional), it orders SA to pre-load its config files and to precompile the patterns, so performance is at least as good as with spamc/spamd setup. All Perl modules are pre-loaded by a parent process at startup time, so forked children need not re-compile the code, and can hopefully share some memory for compiled code;
detailed timing breakdown report for each passed message (with log level 2 or higher);
tunable number of content filtering processes to operate at peak aggregate mail throughput and without wasting more memory than is useful (must be supported by MTA as well, e.g. with Postfix filter setup or dual-sendmail, not with milter or Postfix proxy);
supports SMTP and LMTP server-side and client-side service extension PIPELINING (client side since 2.5.0);
provided utility amavisd-nanny and an associated BerkeleyDB database for a quick-overview monitoring in real time of amavisd child processes;
provided utility amavisd-agent and an associated BerkeleyDB database to provide access to numerous SNMP-like counters and gauges in real time;
see also file README.performance.

-- interfaces to MTA

accepts mail from MTA either via SMTP (fully rfc2821-compliant), or via LMTP (rfc2033), or through a Unix pipe or inet socket from a helper program;
forwards mail via SMTP or LMTP over a Unix or INET or INET6 socket, or by a pipe to some external command (sendmail mail submission program, or its look-alike), or by telling a helper program the mail is to be accepted or not (e.g. with sendmail milter); an optional forwarding method can produce BSMTP files;
in a SMTP or a LMTP relay configuration the amavisd-new can be located on a separate host from the MTA host, and several MTAs may share a single amavisd-new service;
similarly the sendmail/milter allows amavis_milter + amavisd-new to be separated from the MTA host (in sendmail.mc use: INPUT_MAIL_FILTER(`milter-amavis',`S=inet:port@hostname,F=T,T=S:10m;R:10m;E:10m')dnl and start amavis-milter with -p inet:port@0.0.0.0 instead of -p path_to_unix_socket; thanks to Dibo for the tip);
pass reject reason from the MTA on the output side back to MTA on the input side;
proper (non-)delivery status notifications (DSN), compliant with rfc3462 and rfc3464 (ex rfc1892/rfc1894);
informative MTA log entries, especially in the most common SMTP relay setup;
amavis internal log id (am_id) reported in log entries, passed to MTA in SMTP response, and included in a DSN notification (since 2.5.0), making easier to match MTA and amavisd log entries;
supported MTA configurations:
- Postfix supported and thoroughly tested (advanced content filtering model);
- dual-sendmail and other dual-MTA configurations (any MTA type including qmail) with amavisd-new relaying between them (SMTP) is the recommended setup (for speed and flexibility) with other mailers;
- sendmail libmilter interface supported and tested (header rewriting or adding address extensions is only available when using Petr Rehor's amavisd-milter helper program in place of the one provided in the package;
- Exim v4 supported;
- Exim v3 via helper program mostly works, but is not recommended due to deficiencies in the amavis.c helper program or its interfacing with Exim;
- sendmail in the traditional setup with amavis helper program (amavis.c) called as a local delivery agent is still available for compatibility. Please set $forward_method = undef in amavisd.conf if this method is used; the dual-MTA setup with sendmail is preferred;
- since amavisd-new-2.0 a Postfix SMTP extension command XFORWARD is supported on input and output, making available extra information about original SMTP client to amavisd;
- Courier is supported by applying a patch provided in the package;
- qmail interface (qmqpqq) is provided by applying a patch provided in the package;
- Postfix SMTP transparent proxy mode may work for low-traffic sites, but is not a recommended or supported interface method;

-- user individuality, quarantine

can specify subgroups of users who want to receive viruses or spam (with alert header fields added, contents optionally pushed to an attachment, notifications can still be generated);
optionally add address extensions to recipient addresses of mail carrying viruses or spam, facilitating placement of such mail in separate folder at the final delivery stage: e.g. user@domain -> user+spam@domain
can split a message (by clustering to recipient groups of equal treatment) if not all recipients in a multi-recipient message require the same header insertions/deletions/edits, or the same mail body modifications by external sanitation programs or external programs for adding disclaimers (such external programs are supported since 2.5.0);
quarantine options: save to individual files, save to a local mailbox file, save to BSMTP files, save to SQL (no arbitrary size limitations or large memory requirements), pass to MTA for delivery to a quarantine mailbox (possibly remote [not advisable with sendmail/milter unless steps are taken to avoid loop]);
added headers in quarantined messages preserve envelope addresses and quarantine id and facilitate releasing quarantined messages; added headers also identify the reason for quarantining (virus name or spam report);
provided utility amavisd-release to request (from an amavisd daemon) releasing a message from a quarantine; (or use third-party MailZu for managing a quarantine);
can suppress quarantining if spam score is above a configured level;
quarantining can also be specified for passed clean messages, providing an archival capability;

-- anti-virus

optionally decodes/unpacks/decompresses the following formats: MIME, uuencode, xxencode, BinHex, compress, gzip, bzip, bzip2, zip, 7-zip, freeze, lzop, tar, cpio, rpm, deb, rar, arc, arj, zoo, lha(lzh), tnef, ole, cab;
includes support for approx. 40 AV scanners off-the shelf (see file amavisd.conf, variable @av_scanners, for the list);
easy support for command-line virus scanners (new entries are easily described and added to the list @av_scanners, file amavisd.conf);
virus scanners can be split in two lists: @av_scanners and @av_scanners_backup, the second list is only consulted if all primary scanners fail;
supports daemonized virus scanners (e.g. clamd, Sophie, Trophie, DrWebD, FRISK F-Prot, Panda pavcl -tsr) and scanners accessible via Perl modules (SAVI-Perl, Mail::ClamAV);
supports specialized scanners (like a jpeg-scanner), which can check for just one aspect of mail validity and can report malware, but do not vouch for other aspects of a message, so as not to preclude other virus scanners from running;
can specify recipient for whom virus checks need not be performed, fully per-user configurable even with multi-recipient mail;
can specify recipients who want to receive viruses (with alert header field added, contents optionally pushed to an attachment), fully per-user configurable even with multi-recipient mail;
optionally add address extensions: e.g. user@domain -> user+virus@domain, if virus detected and desired to be passed; fully per-user configurable even with multi-recipient mail
can specify final_virus_destiny: reject, bounce, discard, pass (defaults to discard, with optional notification);
bounce suppression: does not accuse innocent users of sending viruses if sender address is believed to be faked;

-- anti-spam

can favour (soft-whitelist) envelope senders (since 2.0) by lowering computed spam score according to recipient's individual wishes;
pen pals soft-whitelist: can favour envelope senders (since 2.4.2) by lowering computed spam score if incoming message can be associated (through SQL logging database) with a previous outgoing message from a local user to the current sender; starting with 2.5.0 recognizes also a reference to a Message-ID of a previous mail;
can blacklist (or soft-blacklist) envelope senders or domains whose messages should be considered spam;
bypass_spam_checks: can specify users or subdomains (envelope recipients) for whom spam checks need not be performed; with proper per-recipient handling of multi-recipient mail;
spam_lovers: can specify users or subdomains (envelope recipients) who want to receive spam, with alert header line added, contents optionally pushed to an attachment; with proper per-recipient handling of multi-recipient mail;
optionally add address extensions (known as plus-addressing): e.g. user@domain -> user+spam@domain, if spam is detected and is desired to be passed; with proper per-recipient handling of multi-recipient mail;
can specify final_spam_destiny: reject, bounce, discard, pass (defaults to bounce, but DSN is suppressed above a cutoff level);
can send spam notifications to spam admin, which include the full Mail::SpamAssassin report; these notifications can be restricted to reporting only spam from internal hosts, through the use of policy banks;
spam headers are inserted on a per-user basis according to their tag/tag2 level settings; this means that a multi-recipient message is split into clusters of recipients with same settings if needed (not available with milter interface). This permits per-recipient individual settings, while still being efficient for multi-recipient messages;
SpamAssassin check is called only once per message regardless of the number of recipients, all header editing and actions taken is then done by amavisd-new for each recipient individually, based on its settings;
bounce suppression for high-scoring spam, or disabled completely.

-- other

customizable notification messages based on simple macro expander in the spirit of m4 -- see README.customize;
versatile lookup mechanisms, described in README.lookups, to be used with virus_lovers, spam_lovers, bypass_checks, local_domains, inet_acl, ... The lookup tables supported are: Perl hash, access control list, Perl regular expressions list, SQL lookups, LDAP lookups;
non-delivery notification are not sent to mailing lists (mail with Precedence: bulk or list, or with a List-Id (rfc2919) header field);
meticulous error checking and handling;
well-commented code;
policy banks -- since 2.0 the whole set of configuration settings may be switched/overlaid, based on incoming TCP port number or original SMTP client IP address (obtained by XFORWARD Postfix SMTP extension command);

Tips and FAQ

Tips and FAQ -- troubleshooting and reporting problems

If running it for the first time, or when encountering a problem, try to start the daemon as a non-detached process with full logging to the terminal: # amavisd debug
When reporting a problem, please include all relevant information. Here is a checklist:
- full amavis version, e.g. amavisd-new-20030616-p10 or amavisd-new-2.4.1; version is logged at startup; more recent versions include the version number in the 'Usage' text, e.g. by amavisd help; or do a grep 'myversion =' /usr/local/sbin/amavisd
- which MTA is being used; running chroot-ed? version of Perl; other information written to the log file at daemon startup time is often useful;
- a description of the problem;
- amavisd log, preferably as the output of amavisd debug or at $log_level 4 or 5 from a syslog file. In certain cases processing of one message may take several minutes, and may be intertwined with log entries from other simultaneously running amavisd child processes. If this is the case, use grep(1) and collect all log entries pertaining to the same thread (some possibly much older than the rest) and submit only these, e.g. egrep '$32036-01$' logfile;
- if hiding certain information from the log, such as e-mail addresses, host names, etc, please do it in such a way as to keep relevant information;
- don't remove timestamps, at least keep minutes and seconds;
- if it looks like the problem is related to MTA, include MTA's log, preferably merged by time with amavisd log (e.g. directing syslogd to write both to the same file);
- if a problem appears to be with SpamAssassin (timeouts, SA configuration problems, some test not working, ...), then run amavisd non-daemonized: amavisd debug-sa. Always check the syntax of SA configuration file after making changes to it, e.g.: su - vscan -c 'spamassassin --lint' The proper forum for general SpamAssassin questions is the SAtalk mailing list; versions of SA older than 2.63 are not recommended;
- here is a sub-checklist of things to do when some amavisd child process seems to be burning CPU without noticeable progress:
  1. start with the output of ps(1), see which process is burning CPU without any obvious progress -- see total elapsed CPU time used so far, and current CPU percentage used by each amavisd child process. Note the PID of the suspect processes(es);
  2. using lsof -p pid, see what files the process has open, especially the current temporary directory holding the mail being processed, and its subdirectories. Go to that directory and examine the mail being processed (email.txt) and its parts. Copy interesting files to a safe place so that you do not lose evidence if amavisd-new happens to time out and delete them;
  3. run strace -t -p pid (or truss(1)) and try to guess what the process is doing. Is it accessing any files? Working on some SA database? (manually doing a Bayes expiry may be beneficial). Is some virus scanner (e.g. SAVI-Perl) busily creating temporary files? ...
  4. kill the process, restart amavisd with debug or rised log level, and feed it the same file (a copy of email.txt) that you saved on step 2, e.g.: sendmail -i postmaster < email.txt Does the same thing happen again? This time the debug output may show further useful information.

Tips and FAQ -- general

The directory specified by $TEMPBASE (typically /var/amavis) is holding temporary directories where mail is being unpacked and checked. Each amavisd-new child process keeps its own temporary directory and works inside it. This means one normally sees $max_servers temporary directories at any time. The temporary directory (with name like amavis-20020924T181627-11984), the file email.txt within it, and the subdirectory parts, are reused (for speed) for subsequent mail checks within the lifetime of a child process, and are only deleted after the child process has done its share of mail checks (after $max_requests).
If amavisd process is killed or reloaded (HUP) (or in an unlikely event of abnormal child death), temporary directories may be left undeleted; this is normal and mail is not lost; actually amavisd may be killed at any time without being in danger of losing mail, as amavisd never takes delivery responsibility away from MTA by acknowledging mail reception without first getting it acknowledged by the second MTA instance. The worst that can happen is a message gets delivered twice, but this is unlikely in practice.
The consequence is that if amavisd is restarted or reloaded often, or if there is some peristent underlying software or hardware problem (perhaps triggered by decoding or analyzing a particular mail message) there may be a growing set of leftover temporary directories, one set of $max_servers directories for every restart, or one directory for every unhandled child process death. Please select and delete these old temporary directories, e.g. using a find(1) Unix command. To be sure one does not delete the still active directories, it is recommended to delete them when amavisd is not running, e.g. just before starting it. The following command does that for example. The -mtime +1 option is just a (non-foolproof) safeguard in case amavisd is currently running, and may be left out:
```
  find /var/amavis -type d -name 'amavis-20??????T*' \
    -prune -mtime +1 -exec rm -rf {} \;
```
There are two regular reasons why temporary directories are intentionally left behind -- preserved to make their later inspection possible:
- after a one-shot debugging is triggered by envelope sender address matching @debug_sender_acl;
- for versions before 2.0: after processing a mail which violated one of the nesting or size limits (resource limits in file amavisd.conf). In this case the X-Amavis-Hold: reason header field is inserted in mail (upon which MTA may be instructed to react), and the following log entry is produced:
  NOTICE: Evidence is to be preserved: ...
When temporary directory is intentionally left undeleted, there is an accompanying message logged: 'PRESERVING EVIDENCE ...'.
If temporary directories are being left behind all the time, and the above explanations do not cover it, there must be something wrong with the configuration or the environment. Increase the $log_level and check the log for the reason why files not being deleted. This should not be happening, find out the problem and fix it, do not sweep it under the carpet by just automating deletion of unwanted files.
Do not let too many files accumulate in directories /var/virusmail or /var/amavis or /var/amavis/tmp. Depending on the file system, large number of files in a directory can lead to long processing times for creation or deletion of a file in such directory, potentially affecting amavisd-new throughput drastically. Keep this in mind when spam quarantine is enabled on a busy site! One may prefer to keep virus and spam quarantine in separate directories.
Versions before 2.0: after amavisd reload (or HUP) the daemon process shuts down but does not restart? As the root privileges are dropped after the first start, the reload must be able to do everything as user amavis (vscan). Some situations where this may not be enough:
- configuration file /etc/amavisd.conf too strongly protected;
- changed the value of $inet_socket_bind in the config file;
- logging directly to a file which is too strongly protected (logging via syslog is preferred);
- running in a chroot jail;
Starting with Perl 5.8.0 Unicode support several problems were introduced with interoperability of software components, some of which are not yet ready for handling UTF-8 character encodings, dealing with character codes above 255 and distinguishing between byte-text, character-text and binary files. Problems of this sort are most pronounced in Red Hat 8.0 Linux distribution, as it sets locale by default to use UTF-8 encoding (run command locale to check the settings). One may see Malformed UTF-8 character warnings, possibly also Bad RFC822 field name 'C0c0\000\000T-Type' (upgrade MailTools Perl module (Mail::Internet) to 1.58 if this error is observed) or panic: swash_fetch. When running chrooted, the swash_fetch panic can also be caused by missing unicore files in chroot jail - see README.chroot .
Awareness and solutions to some of these problems appeared in the 20030314 release of amavisd-new. Also the SpamAssassin people are attacking their share of UTF-8 related problems, and some issues were fixed in SpamAssassin 2.50.

It is best to run amavisd-new in a non-UTF8 locale environment. Either adjust the settings in /etc/sysconfig/i18n (Linux), or set environment variables LANG and LC_ALL to "C" or "en_US" (instead of "en_US.UTF-8") when starting amavisd-new daemon. Depending on the shell used, one may start amavisd-new by (with Bourne or compatible shell):
```
    # su - vscan -c 'LANG=C LC_ALL=C /usr/local/sbin/amavisd'
```
or the long way:
```
    # su - vscan
    $ export LANG; export LC_ALL; LANG=C; LC_ALL=C
    $ /usr/local/sbin/amavisd
```
OpenBSD and NetBSD have a pretty low default setting for max open files. To increase it for the default login group edit the /etc/login.conf, or add the user vscan to the daemon login group which has higher settings. Exceeding the limit can lead to spinning amavisd child processes or Berkeley db 'running out of lockers', often associated with Razor2, Bayes or DCC checks. With debug logging the problem possibly reported as:
```
  CALLING NoMailAudit::check
  Cannot open bayes databases /var/spool/spamassassin/bayes_* R/O:
    tie failed: Too many open files
  razor2 check skipped: Too many open files IO::Socket::INET:
    Bad protocol 'udp' at .../perl5/.../Mail/SpamAssassin/Dns.pm line 409
```
With earlier version of Berkeley db library (libdb) (e.g. V3.3) the following or similar error is sometimes reported:
```
  TROUBLE in check_mail: virus_scan FAILED:
    BDB db_cursor: Successful return: 0, . at ...amavisd line 5162.
```
Namely, a bdb operation fails, but the reported error is 'success'. The problem goes away by upgrading libdb to 4.x.
Manually run the 'amavisd-nanny' utility every once in a while. If there was ever a crashed child process from the previous time the amavisd-nanny was run, the utility would report it, e.g.:
```
PID 00372: 00372-01  went away  0:01:42 =========:=========:=========:=====>
PID 00849: 00849-01  went away  0:02:03 =========:=========:=========:=====>
PID 01490: 01490-01  went away  0:01:55 =========:=========:=========:=====>
```
If any lost processes are reported, their death needs to be investigated, as it can have various consequences, including mail being stuck in an MTA queue until it times out, or the "Lock table is out of available locker entries" failures. Some of the possible reasons for process crashing are:
- a bug in Berkeley database (libdb) or a corrupted database can cause process death at various places: immediately during startup, or when updating nanny state or counters in snmp.db, or in SA when updating or purging a Bayes database. An occasional command: su vscan -c 'sa-learn --force-expire --sync' offers a quick check on the health of a Bayes database and keeps it tidy. When Bayes test is enabled in SA, it is recommended to keep a Bayes database (and preferably AWL as well) on a SQL server, greatly improving reliability and speed of Bayes auto-expiry, compared to having a bdb database. Preferably use bdb version 4.2 or later.
- a bug in Perl module Digest::MD5 can cause a crash soon after receiving a mail message, either when updating entropy pool or when calculating mail body digest; use a recent version of Digest::MD5, and check that its installation self-test is successful;
- a bug in Perl module Digest::SHA1 or Razor agents module can crash a process when SA invokes a Razor test; keep both modules recent;
- a bug in uulib (called by module Convert::UUlib) can crash a process while decoding of apparently plain-text message part; keep module Convert::UUlib recent;
- versions of Perl older than 5.8.2 are known to be able to cause trouble, either due to bugs in the Perl itself, or its incompatibility with some Perl modules; keep Perl recent if possible;
There are several problems with older versions of the Berkeley db library (libdb).
Some sites experience a logged Berkeley db library (libdb) failure:

"Lock table is out of available locker entries". Seems like the same problem manifestation can have more than one cause.

DBD incorrectly compiled with DB_PRIVATE environment option
A telltale: the command db_stat-4.2 -c -h /var/amavis/db fails with:
```
  Berkeley DB library is configured to support only DB_PRIVATE
  environments: Invalid argumentpen: /var/amavis/db
```
Michael W Cocke writes: the reason DB_PRIVATE was enabled is that SuSE 9.1 ships with BDB built wrong! Download BDB 4.2.52 from sleepycat (specifically that version because A LOT of the apps that SuSE 9.1 ships with are hardcoded to that specific version). Compile with --enable-cxx and NOT posixmutexes! Then install it as usual. You make have to rebuild BerkeleyDB as well. I have no idea if SuSE 9.2 has the same problem.
Use recent versions of bdb library and the associated BerkeleyDB Perl module

For bdb the versions 4.2 or 4.3 (or later) and their recent patch-revisions should be used. John Beranek writes: When I was having the problem I was in fact using BerkeleyDB 0.25. Ever since I installed BerkeleyDB 0.26 off CPAN I've not had the problem.

bdb 4.1 as used by amavisd can severely cripple the performance of amavisd

(processes waiting on 'select', elapsed times start to go up). Switching to bdb 4.2 solved the problem as reported by Andy Dills using FreeBSD 8.0.

Some system ship with open file descriptors limit quite low, which can have different manifestations.

Paul Jacobson writes: It seems the main problem was that the server was exhausting open file descriptors, and despite changes to login.conf to allow unlimited open files for the amavisd user the limit remained at 1772. Some googling on "openbsd open file descriptions" revealed that changes to the open file limit had no effect with the default kernel setting of maxusers=32. I've recompiled the kernel with a maxusers=256 (this may be excessive) and the amavisd user now has an open file limit of 13196. This has solved the problems I was experiencing with DCC and Razor2 on some emails [and lead to "Lock table is out of available locker entries"].

Run the db_stats utility every once in a while,

checking for bdb resources being depleted. The db_stat should be run (manually or perhaps from cron, writing to a time-stamped log file) especially when timeouts and helper program failures happen: db_stat-4.2 -c -h /var/amavis/db See http://www.sleepycat.com/docs/utility/db_stat.html for its man page. The -h option should match the $db_home variable in amavisd.conf, which is typically: $db_home = "$MYHOME/db";

As a last resource, the use of bdb may be turned off.

The $enable_db=0 disables use of bdb altogether, falling back to memory-based cache and loss of amavisd-nanny and amavisd-agent functionality.

Tips and FAQ -- mail transfer agents (MTA)

For mail setups such as with sendmail/milter where messages do not pass through amavisd-new, header fields can not be dynamically inserted, recipient addresses can not be dropped or address extensions added, because of the limitations of the supplied helper programs. Petr Rehor's version of amavisd-milter (see Contributed and related software) using a newer AM.PDP protocol aleviates this limitation to some extent. The limitation does not apply to Postfix, Exim v4, or dual MTA (any MTA type) setups.
sendmail/milter: because -odd option (deferred delivery mode) is now used to submit notifications in the sendmail/milter setup, sendmail must be told to periodically process the clientmqueue queue. See sendmail options -q and -qp. A command like sendmail -Ac -qp1m may be placed in the sendmail init script, if not already there;
Postfix: versions before Postfix 2.0 had two minor problems with its lmtp client - it may fail to parse its parameters and obtain port number, and it may lowercase mail addresses. If these problems show, one may stay with smtp protocol, or upgrade Postfix to 2.0.
Postfix versions before 20040616: space in HELO commands could end up in XFORWARD command, triggering tarpitting (artificial slowdown) in amavisd; fixed in Postfix 2.1.4;
Is sendmail milter failing, producing the following (or similar) amavisd log: RX_tempdir FAILED, retry: Invalid temporary directory '\000\000\000\rO' ? It indicates a setup error, sendmail is trying to talk to amavisd daemon directly. There are 2 sockets when using sendmail in milter setup: one for sendmail -> amavis-milter, and one for amavis-milter -> amavisd communication. See README.milter.
The Postfix Before-Queue Content Filter setup, also known as smtpd_proxy setup, is not a supported or recommended setup with amavisd-new, which is not a transparent SMTP proxy by design. See caveats in README_FILES/SMTPD_PROXY_README. This setup might work amavisd-new for low-traffic sites which do not use authentication, but is not recommended.

Tips and FAQ -- virus scanners

If virus notifications is observed claiming the virus originator is <?> or <?@some.domain> and sender notifications are not sent, this is not a bug, but a feature -- see $viruses_that_fake_sender_re regexp lookup table. The original idea comes from Furio Ercolessi: as some viruses tend to use forged or corrupted envelope sender or 'From:' addresses, we try to determine the true virus sender, and if we can not do that, we avoid accusing innocent users of sending viruses by not generating sender notifications;
The recommended Sophie configure option is --enable-error-strings . Always start Sophie using a full absolute path.

Tips and FAQ -- spam scanners (Mail::SpamAssassin)

Recommended versions of SpamAssassin are 2.64 and 3.0.1 or later.
Does SpamAssassin observe settings in its configuration file local.cf?
- SA does observe all settings in its configuration file, but not all of them have effect on the mail being checked, as amavisd-new does its own decisions based on spam score (hits) (so for example required_hits has no effect - use tag/tag2/kill amavisd-new settings instead), and does its own header editing, and body is not modified. Read on for related information.
SpamAssassin has configuration options to modify mail body and header, but they seem to be ignored.
- amavisd-new does not modify mail body or lets SA do it (with the exception of defanging, introduced with amavisd-new-2.0). All mail (header) editing is done by amavisd-new and not by SA. Even though SA does observe options in its configuration file to rewrite mail body and modify mail header, the result is purposely not used by amavisd-new. There are two reasons for that: SA is only called once per message regardless of the number of recipients, and secondly, to be able to offer a guarantee the mail body will not be altered, This means the per-recipient handling of mail relaying and header editing needs to be done entirely in amavisd-new, as there are no provisions in SA to analyze mail once and then prepare different modifications for different recipients based on the same spam analysis. It is a tradeoff: speed for multi-recipient mail versus the full per-recipient flexibility. It would make no sense to fully duplicate the spamc/spamd functionality in amavisd-new. If you need such features, just disable calling SA from amavisd-new, and use the spamc/spamd or other back-end interface to SA.
What's the better way to specify SA options? Directly in amavisd.conf, or in the SA's local.cf file?
- There is hardly any choice. Options to control trigger levels for spam (tag/tag2/kill level) must be in amavisd.conf, as it is the amavisd that decides what to do with a mail based on the score level as calculated by SA. The required_hits in local.cf has no effect on mail delivery or tagging.
- Similarly the header insertion/editing options must be specified in amavisd.conf. It is the amavisd that is editing the message by itself. Header and body rewriting options in SA have no external effect, as amavisd does not use the modified message as prepared by SA.
- The $sa_local_tests_only must be specified in amavisd.conf, there is no equivalent options in local.cf. The setting affects how SA is called from the application (the API).
- The $sa_auto_whitelist must be specified in amavisd.conf for SA versions older than 3.0, there was no equivalent options in local.cf. Starting with SA 3.0, there is now an option use_auto_whitelist to be specified in local.cf, and the $sa_auto_whitelist is ignored.
- White and blacklisting in amavisd-new and as provided by SA are similar in concept, but different in implementation. Both can coexist, use the mechanism that best suits the needs.
  - The w/b listing in SA works on information from the header (e.g. on the mail author -- the From: header field), and contributes large positive or negative score points to the score being computed.
  - The (hard) w/b listing in amavisd-new works on envelope sender address (i.e. the return-path). If triggered, the call to SA is skipped to save time, as it would not have a chance to overrule the w/b list decision already taken.
  - The soft w/b listing in amavisd-new (the @score_sender_maps, available since 2.0) also works on envelope sender address, but only modifies the spam score as returned by SA, and does not bypass calling SA.
- SA sees and observes all options in local.cf. It is just that some of them have no effect on the mail.
No spam-related headers inserted? Here are some reasons:
- @local_domains_acl is not correctly set. These headers are only inserted for recipients matching @local_domains_acl lookup (or %local_domains or $local_domains_re or field 'local' in SQL lookups);
- headers can only be added or edited when messages pass through amavisd-new. This currently is not the case with sendmail milter setup (using the helper program amavis-milter.c);
- tag level is set too high ($sa_tag_level_defl);
- when SpamAssassin is not being called (disabled, message larger than the $sa_mail_body_size_limit, sender white/blacklisted), or SA returns an empty score e.g. when it times out, the spam score is empty (undefined);
- to make message with spam score above kill_level still pass, either set globally: $final_spam_destiny=D_PASS, or declare recipient a spam_lover.
How to add the spam tags to all inbound messages so that spam score and test information appear in the message header? By reducing the tag level (and keeping tag2 and kill levels high if desired), one may enable spam-related header fields to be inserted to inbound mail (i.e. for recipients matching @local_domains_acl)
- tag level is where X-Spam-Status and X-Spam-Level header fields start to appear (e.g. setting tag level to 0 (or even better to -9999) would turn this on permanently);
- tag2 level is where a message is considered spam as far as mail header fields and adding address extensions are concerned: the X-Spam-Flag: YES header field appears, the X-Spam-Status gets a YES, Subject gets a ***SPAM*** if subject editing is enabled, (optional) recipient address extensions are added;
- kill level is where a message is considered spam and countermeasures are taken: (reject/bounce/discard/pass), quarantine, notify). It is common to set tag2 level the same as kill level, but some may prefer to set kill level even higher, perhaps combined with $final_spam_destiny=D_DISCARD;
Sendmail milter is able to insert header fields as evidenced by X-Virus-Scanned header field. Why can't spam-related headers be inserted?
- In the milter setup the static header fields are inserted autonomously and unconditionally by the amavis-milter helper program itself. Dynamic fields (with spam levels reported etc.) would need to come from the amavisd daemon for each message checked, but the protocol between amavis-milter and amavisd does not support passing this information. Use Petr Rehor's amavis milter helper program instead of the one supplied in the package.
SpamAssassin returns different score or null score or triggers different set of SA rules when called from amavisd-new, as compared to the command-line utility spamassassin on the same message. What is wrong?
- the spamassassin utility program (to be used for testing) needs to be invoked from the same user as amavisd runs under, to make them operate in the same environment and use the same settings and databases;
- if running under chroot, make sure that SA sees the same rules directories and the same databases within the jail, as spamassassin sees outside of the jail. The SA rules directory typically needs to be copied to the jail, the accessibility of /var/amavis/.spamassassin in both cases is usually taken care of by the soft link /var/amavis/var/amavis as described in README.chroot;
- have SA network tests (nonlocal) enabled or disabled equally for both tests (spamassassin --local is equivalent to $sa_local_tests_only=1;)
- if they are still returning different results even when running under the same username/uid, compare the SA debug reports obtained from spamassassin -D -t < message.msg and by sending exactly the same message and running amavisd-new in SA-debug mode: amavisd debug-sa; compare paths to directories used in each, and take notice of possible problems reported by SA, especially during its initialization phase;
- check that the spam test-mail sent by MUA does not got mangled, e.g. wrapped in another header or sent as an attachment; a different mail header leads to large differences between the two cases. See ./test-messages/README on how to properly send a test message without changing it.
- SA is not called when the sender is white- or blacklisted, or when the message body is too large (200 kB by default, configured by $sa_mail_body_size_limit);
- null score is returned if SA times out (30 second timer $sa_timeout),
- null score is also returned if SA can not find or read its configuration file or its score and rules files, e.g. when they are not at the default location. When chrooted, these files need to be copied to a corresponding directory in the jail. Start as: amavisd debug-sa to verify. Additional parameters may be supplied in the argument list of the Mail::SpamAssassin->new call (file amavisd, see Mail::SpamAssassin man page), for example:
```
  DEF_RULES_DIR => '/usr/local/share/spamassassin',
  LOCAL_RULES_DIR => '/etc/mail/spamassassin',
```
If SpamAssassin complains (in the amavisd-new log file):
Failed to create default user preference file /var/amavis/.spamassassin/user_prefs
just create an empty user_prefs file, e.g. with: touch /var/amavis/.spamassassin/user_prefs
Older versions of SpamAssassin, specially those before the 2.53, had a tendency to fall in CPU loop when analyzing certain types of mail messages. These problems have been mostly avoided with SA 2.53, and more so with 2.64. The recommended version of SpamAssassin is 3.1.1 or later.
If it appears that amavisd is spinning during its call to SA, or timeouts are reported by MTA, before spending too much time on investigating, insure that:
- your SA is up to date, upgrade from old 2.5x versions;
- if using Bayes in SA, do a manual bayes db expiration by running sa-learn as user amavis: su amavis; sa-learn --force-expire --rebuild before investigating further;
- if having a large Bayes database, disable opportunistic auto-expiry in the SA config file and do bayes database expiry periodically from cron; see next entry below;
- Perl should not be 5.8.0 or older; the 5.8.2 or later is recommended;
Aborted Bayes auto-expiry runs: if Bayes database checks are used by SA and opportunistic auto-expiry is not disabled, one may occasionally observe a large and prolonged increased load and unresponsiveness of the server and/or a backtrace in the log casued by SA taking too long to execute:
```
  SA TIMED OUT, backtrace: at .../Mail/SpamAssassin/BayesStore/DBM.pm line 619
  ... Mail::SpamAssassin::BayesStore::expire_old_tokens_trapped
```
The situation can be worrying if this happens more than once in a blue moon, as aborted bayes auto-expiry most likely will not succeed even on the next automatic attempt, and each auto-expiry attempt consumes huge amounts of resources (CPU and I/O), especially if Bayes database is on a Berkeley database. There are two possible solutions to the problem:
1. avoid using file-based database for Bayes; moving a database to SQL offers a faster and more reliable back end, and its best advantage is that auto-expiry is an order of magnitude faster. Instructions for converting database to SQL are in SA distribution in subdirectory sql/
2. alternatively, one may disable opportunistic (automatic) Bayes auto-expiry (set bayes_auto_expire to 0 in local.cf, see Mail::SpamAssassin::Conf man page), and do periodic (e.g. nightly) database expiration runs, e.g. from a cron job. Be sure to do it under the same username as amavisd runs under, e.g.: su vscan -c 'sa-learn --force-expire --sync'
If Mail::SpamAssassin is configured to call Vipul's Razor 2.22 to 2.36, it fails because reading its config file (routine read_file in Razor2/Client/Config.pm) produces tainted values. Please upgrade to the current version of Razor2 agents.
On some operating systems the Perl module IO::Socket reports obscure error on unsuccessful connect() in non-blocking mode as: "Invalid argument" in place of the true cause: "Connection refused". The remote service should be checked if available and accessible (firewall rules? network problems?).
Another possible reason for "Invalid argument" reported by IO::Socket when running within chroot jail is some network-related system file missing in a chroot jail, e.g. /etc/protocols, /etc/services, /etc/netconfig, /etc/resolv.conf, /etc/nsswitch.conf, /etc/hosts, /etc/host.conf -- depending on the operating system in use. System debugging tools (debugger, strace, truss) may help to find the cause.
For debugging SQL client side in amavisd, see the DBI man page. An environment variable DBI_TRACE can control the DBI/DBD log, e.g. DBI_TRACE=2=/tmp/dbitrace.log amavisd
Running in a chroot jail and Razor agents complain about "Invalid argument"? Most likely the underlying cause is the same as above.
Seeing NOTICE client broke the connection without a QUIT in amavisd-new logs?
This message usually means that SpamAssassin or one of its external services played jokes on STDIN or STDOUT (like closing it), which is associated with SMTP/LMTP socket in amavisd-new. So what would be an innocent close in a command-line application, is breaking the SMTP or LMTP session. If it happens at the very end before the QUIT, it is innocent, otherwise it may indicate a more serious problem.

Reportedly this happens when SA debugging is enabled ($sa_debug = 1; or starting the daemon with amavisd debug-sa), but goes away when no SA debugging is enabled.

Tips and FAQ -- Net::Server

amavisd-new version 2.3.3 or earlier is incompatible with Net::Server 0.91 or later versions; please upgrade amavisd-new to 2.4.0 or later, or keep Net::Server at 0.90;
If during startup the Net::Server complains that it can not change UID or GID to the required $daemon_user and $daemon_group, please upgrade it to Net::Server 0.90, which has this bug fixed.
Alternatively, one can start amavisd as non-root (only applicable if not running chroot-ed):
```
    # su - vscan -c /usr/local/sbin/amavisd
```
or if some command line options need to be specified, e.g.:
```
    # su - vscan -c '/usr/local/sbin/amavisd -c /etc/amavisd-test.conf debug'
```
After changing $inet_socket_bind in amavisd.conf, the amavisd process must be stopped and restarted. The HUP method causes amavisd to stumble over its feet.
The HUP method of restarting amavisd is deprecated since 2.0. Please use amavisd reload, which works equally well for chrooted or non-chrooted setups.

Security considerations

Security considerations for the host running amavisd-new

amavisd-new accepts mail from MTA, may call external Perl modules and may fork external programs to decompress and decode message, classify its content, then the checked mail is either passed to MTA for delivery, or rejected and quarantined.

Any component of a program that comes in contact with unpredictable and possibly malicious mail/document content, must be careful not to let the content have any uncontrolled effect on the operation of the program, or its environment.

amavisd-new is written entirely in Perl, with taint mode Perl checking enabled. This in itself is a strong argument that the processing within amavisd-new (and Perl modules it calls) is not likely to be subject to buffer overruns, stack smashing, and other problems that are common source of security problems in programs written in languages like C.

Information coming from external world like SMTP envelope information, mail header and body contents, suggested MIME file names, etc., is only used by amavisd-new for operations that do not influence the program environment. For example, names of created temporary files are internally generated and do not depend on suggested file names from MIME header. Mail addresses or other tainted information is never passed through shell to an external program.

The external Perl modules called by amavisd-new have not been thoroughly screened for possible security implications. They still benefit from the Perl environment, and the Perl taint mode checking is not turned off even when other Perl modules are executing, including SpamAssassin if enabled (which is a relatively complex piece of software). Perl modules that deal with decoding and checking of mail contents may be targets of malicious mail content, especially if they include code written in C, like decoding and uncompressing libraries, e.g. zlib and uulib/uudeview (Convert::UUlib).

External programs that get forked from amavisd-new to perform some decoding/uncompressing or classifying task, are the greatest potential threat to the safe operation of the host running amavisd-new. Some of these programs that are used to decode certain archive formats are quite complex, are old or poorly maintained, and/or written by less security conscious authors. E.g. a vulnerability is present in Unix utility file(1) version 3.41 or older. Generally it is advised that external programs are kept up-to-date and that crashes of such programs are reported immediately to their maintainers (after verifying first the version is recent).

There is a tradeoff in deciding whether to call some external decoder: calling it may open a vulnerability at the host running amavisd-new; not calling it (and not decoding certain types of document) may cause virus checker to miss a malicious mail contents, increasing danger for the mail recipient, while reducing risk for the host running checks.

While it may be true that only a powered-down computer, locked in a basement and disconnected from the network is completely secure computer, this is not practical to get any job done. Besides choosing a content filtering program to be written in Perl and using taint mode checks, there are other things one may do to reduce security threats to the computer running content filter:

Never run amavisd-new as root or with other elevated privileges. Use a dedicated username (UID) and group (GID) for the purpose (for example: don't use usernames mail or nobody). Start amavisd-new daemon with su(1) (best), or use command line options -u (and -g) to specify username/group when starting amavisd, or at least specify $daemon_user and $daemon_group in amavisd.conf . Later versions of amavisd-new perform certain checks on its environment and fail to start up if some obvious security flaw is found in current UID of the process or in ownership and protections of the most critical files and directories.
Protect account under which amavisd-new will run, don't let it have a valid password and disallow interactive logins and remote access.
Check settings for external programs like $arc, $unarj, $unrar, $zoo, $lha, $lzop, $unfreeze, $uncompress, $gzip, $bzip2, $cpio, $rpm2cpio, $cabextract, and disable those not trusted (amavisd.conf, or just remove them). Make sure external programs, their configuration files (e.g. /usr/share/magic) or directories where they reside, are not writable by a non-privileged (non-root) user. Normally such files should be owned by root. This also applies to external virus checking programs and their database, and external programs that may be called by SpamAssassin (if enabled).
The same applies to configuration files used by amavisd-new and SpamAssassin, e.g. /etc/amavisd.conf, /etc/mail/spamassassin/*, /usr/share/spamassassin/* . Most importantly these files should not be writable by user (UID or GID) under which amavisd-new is running, they should preferably be owned by root and not (world or group)-writable.
The directories /var/amavis/tmp, /var/virusmail, /var/amavis/db, /var/amavis/.spamassassin, /var/amavis/.razor ($TEMPBASE, $QUARANTINEDIR, $db_home) must be writable for amavisd-new process, and are commonly owned by user and group under which amavisd-new is running. These directories should not be writable by other non-privileged users. It is advised to keep $TEMPBASE in a separate subdirectory (e.g. /var/amavis/tmp) and not letting temporary files be created in the top-level /var/amavis directory.
chroot mode of operation is a very powerful security concept in Unix. amavisd-new can work in a chroot environment since amavisd-new-20021116. This requires that all external programs called by amavisd-new can operate in a chroot file system subtree. Preparing a chroot environment including all required programs with their shared libraries, is highly system-dependent. Using only sockets to communicate with the external world (e.g. SMTP, daemonized virus scanners) simplifies the setup. Unfortunately setting up chroot environment for amavisd-new is not for inexperienced Unix administrators. Besides documentation in README.chroot, recommended reading for setting up chrooted environment for amavisd-new and other components is also: Fairly-Secure Anti-SPAM Gateway Using OpenBSD, Postfix, Amavisd-new, SpamAssassin, Razor and DCC, by Scott Vintinner.
Keep all software up-to-date, especially the external decoders and virus scanners.
Do regular backups, keep file system signatures (e.g. with Tripwire).
Choosing a platform less widespread and popular may help: Alpha or Sparc CPU instead of Intel, *BSD or Tru64 or Solaris instead of Linux (not to mention Windows) may help. This may be considered security through obscurity, but any additional obstacle can help.

Security considerations for mail clients being protected

Running a virus checking content filter for each mail before it reaches the mail reader is an important line of defense against virus outbreaks and in protecting the (possibly not security conscious) recipients, or their mail reader programs or computer environment.

Not all malware is passed by e-mail. Several viruses or worms use multiple mechanisms to propagate, including WWW, sharing disks or through peer-to-peer 'contents' sharing, social engineering, or even a memory key or a CD brought-in in a pocket or distributed by magazines and software publishing houses may bring in a virus;

Content filtering mailer can not protect internal hosts unless incoming SMTP (TCP dst port 25) is restricted at the firewall to official mailers only. Similarly external world deserves protection from possibly infected internal hosts, so outgoing SMTP (TCP dst port 25 again, outgoing this time) needs to be restricted to official mailers. (Use standard tcp port 587 for mail submission from roaming users.)

Similarly, if mail readers can fetch mail from external mailboxes (POP3, IMAP), the SMTP mail gateway can not protect them. One solution is to provide a centralized fetchmail service to users that need access to external mailboxes, and feed such mail to the regular content filtering mailer, while blocking other unofficial access to external POP3 and IMAP servers at a firewall.

Even in e-mail, malware may be carried in encrypted or scrambled form, or simply as a plain text, using social engineering techniques to persuade recipient to fetch or activate malware.

It is not possible to prevent user shooting himself in the foot, or to prevent a dedicated person to transfer malware. There is a tradeoff in keeping e-mail useful, and protecting against threats.

The first line of defense (mail content filtering, firewall) must be complemented by defense mechanisms at the local user's desktop computer. This includes virus scanners run on PCs, keeping software up-to-date, doing backups, and educating users.

Malware does not have to play by the rules. Nothing prevents malware from generating a syntactically incorrect mail, to send it directly to some host ignoring MX and A records, to supply forged SMTP information or forged mail header, to poison DNS, perhaps even to use forged source IP address.

Content filter with virus scanner tries to decide if the mail under consideration will, or can, cause any bad effects on the recipient computer, often without knowing what mail reading software or what computer is used by recipients. This implies that while some mail may be decoded (by adhering to standards) into a harmless text, it might be decoded by some broken MUA or archiver into a virus or exploit, or trigger a MUA bug or vulnerability during decoding, or during displaying a message. External archivers/unpackers called by amavisd-new may be relatively easy to trick into not extracting certain archive members, thus hiding malicious code. See Malformed email project, Bypassing content filtering whitepaper, Declude's list of vulnerabilities, NISCC Vulnerability Advisory 380375/MIME. CAN-2003-1015

Solving this problem would require content filter with virus scanner to emulate all known (and unknown?!) mail readers in the way they respond to malformed mail. While amavisd-new and other content filters try to anticipate some common problems, especially the ones practiced by currently active viruses, there is no guarantee that this approach is always successful.

Even now there are combinations of viruses and virus scanners (e.g. Yaha.K + Sophos) that fail to be detected due to a malformed MIME header, which gets decoded differently (and correctly, considering standards!) by MIME::Parser, yet certain mail readers decode it differently, forming a virus. It often helps to use more than one virus scanner (e.g. clamd along with some commercial virus scanner).

RFC 2046 defines a way to split sending one document into several e-mail messages, which can then be reassembled (automatically or manually) by MUA. The Content-Type value to look for is message/partial (and similarly: message/external-body). Checking mail fragments individually for viruses can not reliably detect viruses, which only get reassembled into a recognizable form by the recipient's mail reader. Most virus scanners at the MTA level (including amavisd-new and all other variants of AMaViS*) check each mail independently from other messages, so the only protection to this threat is to ban these MIME content-types (see $banned_filename_re setting in amavisd.conf), or by disabling auto-reassembly at mail readers, or running a virus checker tightly associated with MUA.

Blocking the MIME content type message/external-body may sound useful, although the mechanism is not much different from letting user freely browse the web or fully interpret HTML mail messages, so if the later is allowed, it probably does not make sense to treat message/external-body differently.

Protection against denial-of-service (DoS) attacks

Because amavisd-new tries to recursively unpack and decode each mail as deeply as possible, this may be abused by malware. The so-called mail bomb, e.g. 42.zip or bzip2 bomb are examples of such malware. Such mail message, when fully decoded, can exceed available disk size several times, or consume a lot of time for decoding. Unless decoding is stopped at an earlier stage, it could cause the message checking to be retried over and over again, each time either hitting the disk full condition, or exceeding the allowed time limit. Note that mail bombs are targeting mail content filters, and are normally not a threat to mail clients (MUA), unless they carry a virus as well.

amavisd-new has several configurable mechanism for limiting the amount of space consumed during decoding - see resource limits in file amavisd.conf. When message decoding exceeds the storage quota, the decoding stops, the virus scanning is not performed to protect the virus scanner, but a header field is inserted, telling MTA it may place the message 'on hold', or reject it, or just pass it - the action depends on MTA configuration. This works well with Postfix, but may not be configurable with some other mailers.

Since amavisd-new-20030616-p8 a string '***UNCHECKED***' is inserted into Subject. Since amavisd-new-2.0 such passed mail is also wrapped into a MIME wrapper (defanged), prepended by a warning message.

Protection against mail loss

When a content filter is positioned in relation to MTA as a mail relay (or proxy), accepting mail from MTA, and passing all checked mail to MTA for final delivery (e.g. Postfix, or dual-sendmail setup), there are only two possible approaches that can prevent mail loss when unpredictable things happen:

committing messages to secondary storage and keeping evidence, e.g. properly implementing a queueing mechanism, as MTAs do it, or
confirming mail reception to the input-side MTA only after the forwarded mail reception has been confirmed by the MTA on the output side (or a non-delivery notification sent). This approach is used by transparent SMTP proxy, or by cooperating SMTP server and client.

amavisd-new chooses the second approach. Some alternative mail content filtering solutions based on Perl module Net::Server::SMTP can not guarantee not losing mail under certain circumstances, because they confirm mail reception before being in a position to ensure a mail delivery or bounce.

Besides taking care of not losing mail, it is important the mail contents is not unintentionally changed, as could happen for example when disk is full, or communication or I/O errors occur. amavisd-new is thorough in always checking the status of operations, e.g. all I/O operations, creating/deleting/writing to files, calling external programs, checking all SMTP response codes, etc. When a problem occur, amavisd-new tries to produce an error report in its log file that is as informative as practical. When the situation can not be corrected, a temporary failure (EX_TEMPFAIL, or 450 SMTP response) is generated, telling MTA to try again later, hoping for the postmaster to notice stuck mail if the problem keeps reoccurring.

The amavisd-new policy is to either deliver the mail, or to make sure the sender gets a non-delivery notification. It is possible to configure amavisd-new to disobey this policy for certain unwanted contents, e.g. only to quarantine spam and not generate bounces. See README.policy-on-notifications and choose your policy. Since amavisd-new-2.0 the default policy for viruses is to discard them after quarantining. Configurable options are provided to reduce undesired bounces on spam: @spam_dsn_cutoff_level_maps (with $sa_dsn_cutoff_level) and @spam_dsn_cutoff_level_bysender_maps.

mm
Last updated: 2018-10-08